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3.2.2 Expectation 

If you have a collection of numbers CL\, CL 2 , ■ ■ ■ , CLn> their average is a single number that 
describes the whole collection. Now, consider a random variable X. We would like to define its 
average, or as it is called in probability, its expected value or mean. The expected value is defined 
as the weighted average of the values in the range. 


Expected value (= mean=average): 

Definition 3.11 

Let X be a discrete random variable with range Rx = \ x h x 2i x 3i • • -j (finite or 
countably infinite). The expected value of X, denoted by EX is defined as 

EX = ^ x kP(X = ajjfe) = ^2 x kPx{ x k)- 

XkGRx Xk^Rx 


To understand the concept behind EX, consider a discrete random variable with range 
Rx = \ x h x 2i x 3i . . . }. This random variable is a result of random experiment. Suppose that 
we repeat this experiment a very large number of times N, and that the trials are independent. Let 
N\ be the number of times we observe X\, N2 be the number of times we observe X2, ■■■■, N k 
be the number of times we observe X k , and so on. Since P(X = X k ) = Px \ x k)i we expect 
that 


Px( x i) 


Ni_ 

N ’ 


Px( x 2) 


N 2 
N ’ 


Px{xk) 


N k 
N ’ 


In other words, we have N k ~ NPx (x k ) . Now, if we take the average of the observed values 
ofX, we obtain 

Average = 
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x 1 NPx(x 1 )+X2NP x {x 2 )+x 3 NPx(x 3 )+... 


= xiPx(xi) + x 2 Px(x 2 ) + x 3 P x (x 3 )+.. . 

= EX. 

Thus, the intuition behind EX is that if you repeat the random experiment independently N 
times and take the average of the observed data, the average gets closer and closer to EX as N 
gets larger and larger. We sometimes denote EX by [Lx ■ 


Different notations for expected value of X: EX = -E7[JT] = E[X) = [Lx ■ 


Let's compute the expected values of some well-known distributions. 


Example 3.11 

Let X ~ Bernoulli(p). Find EX. 

• Solution 

o For the Bernoulli distribution, the range of X is Rx = {0,1}, andPx(l) = P 
and Px (0) = 1 — p. Thus, 

EX = 0 • Pjf(0) + 1 • Px(l) 

= 0 • (1 -p) + 1 'P 

= p 


For a Bernoulli random variable, finding the expectation EX was easy. However, for some 
random variables, to find the expectation sum, you might need a little algebra. Let's look at another 
example. 


Example 3.12 

Let X ~ Geometric(p). Find EX. 

• Solution 

o For the geometric distribution, the range is Rx — {1,2,3,...} and the PMF is 
given by 

Px(k)=q k ~ 1 p, for k = 1,2,... 
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where, 0 < p < 1 and q = p — 1. Thus, we can write 


EX — Y2x k eRx X kPx{%k) 

= E ZM-'p 

= pEt 1 kg k ~ 1 - 


Now, we already know the geometric sum formula 


oo 


fc =0 


1 

1 — X ’ 


for x < 1. 


But we need to find a sum ^ kq k 1 ■ Luckily, we can convert the geometric sum 
to the form we want by taking derivative with respect to X, i.e., 


d 

dx 


OO 


fc=o 


d 1 

dx 1 — cc ’ 


Thus, we have 


^kx k ~ l 

k =0 


1 

(1 — x) 2 ’ 


for tc < 1. 


for tc < 1. 


To finish finding the expectation, we can write 

EX = v Er=iV-' 

1 


= p 


(1-9) 2 


= 

_ L 

p' 


So, for JC ~ Geornetric(p). EX = r . Note that this makes sense intuitively. 

The random experiment behind the geometric distribution was that we tossed a coin 
until we observed the first heads, where P(H) = p. Here, we found out that on 
average you need to toss the coin — times in this experiment. In particular, if p is 

small (heads are unlikely), then — is large, so you need to toss the coin a large number 

of times before you observe a heads. Conversely, for large p a few coin tosses usually 
suffices. 


Example 3.13 
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Let X ~ Poisson^ A). Find EX. 

• Solution 

o Before doing the math, we suggest that you try to guess what the expected value 
would be. It might be a good idea to think about the examples where the Poisson 
distribution is used. For the Poisson distribution, the range is Rx = {0, 1, 2, • • •} 
and the PMF is given by 

Px(k) = 6 , for k = 0,1,2,... 

Thus, we can write 

EX = Yh Xk e Rx x kPx{xk ) 

_ V '' 00 he- x \ k 
~ 0 ^ k\ 


-A v^°° A fc 

X/k= 1 1)1 


A A^ +1 ) 

e 0 j\ 

( by letting j — k — 1) 

\e~ x V°° Y 

Ae 2^j= 0 j\ 


Xe~ x e x 

( Taylor series for e x ) 


= A. 

So the expected value is A. Remember, when we first talked about the Poisson 
distribution, we introduced its parameter A as the average number of events. So it is 
not surprising that the expected value is EX = A. 


Before looking at more examples, we would like to talk about an important property of 
expectation, which is linearity. Note that if X is a random variable, any function of X is also a 
random variable, so we can talk about its expected value. For example, if Y = dX + b, we can 
talk about EY = E\aX + b\. Or if you define Y — X\ + X-2 + • • • + X n , where Xj s 
are random variables, we can talk about EY = E[X\ + X 2 + • • • + X n \. The following 
theorem states that expectation is linear, which makes it easier to calculate the expected value of 
linear functions of random variables. 
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• E[X i + X2 + • • • + X n ] — EX 1 + EX 2 + • • • + EX n , for any set of 
random variables Xi, X2, • • • , X n . 


We will prove this theorem later on in Chapter 5, but here we would like to emphasize its 
importance with an example. 


Example 3.14 

Let X ~ Binomial(n,p). Find EX. 

• Solution 

o We provide two ways to solve this problem. One way is as before: we do the math and 
calculate EX = '^ 2 Xk ^R x x kPx { x k) which will be a little tedious. A much 
faster way would be to use linearity of expectation. In particular, remember that if 
Xi, X2, . . . , X n are independent Bernoulli (p ) random variables, then the 
random variable X defined by X = X\ + X2 J r- • • PX n has a 
Binomial(n,p) distribution. Thus, we can write 

EX = E[X 1 + X 2 + • • • + X n ] 

= EX 1 + EX2 + • • • + EX n by linearity of expectation 

= p + p~\ -b p 

= np. 

We will provide the direct calculation of EX = '^2 Xk eRx x ^Px i x k) i n the 
Solved Problems section and as you will see it needs a lot more algebra than above. 
The bottom line is that linearity of expectation can sometimes make our calculations 
much easier. Let's look at another example. 


Example 3.15 

Let X ~ Pascal(m,p). Find EX. (Hint: Try to write X = X\ + X2 + • • • + X m , 
such that you already know EXi.) 

• Solution 

o We claim that if the Xi' s are independent and X t ~ Geometric(jp ), for i = 1, 
2, • • •, m, then the random variable X defined by X = X\ + X2 + • • • + X m 
has Pascal(m,p). To see this, you can look at Problem 5 in Section 3.1.6 and the 

discussion there. Now, since we already know EXi = ^, we conclude 

EX = E[X\ + X 2 -\ - b X m \ 
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= EX i + EX 2 + • • • + EX r 


by linearity of expectation 


1+1 + 
p p 


Again, you can try to find EX directly and as you will see, you need much more 
algebra compared to using the linearity of expectation. 


<— previous 
next —> 


https://www.probabilitycourse.com/chapter3/3_2_2_expectation.php 



